ABSTRACT
BACKGROUND: The transition to telemedicine at the onset of the COVID19 pandemic revealed a need for new clinical communication skills. Previous standardized assessments indicate that telehealth requires a distinct skill set that most residents and physicians have not yet mastered. We created a competency-based experiential training program using announced standardized patients (ASPs) to train and assess clinicians in virtual practice. METHODS: From June to October 2021, clinicians participated in a video tele-visit in which an ASP portrayed one of two cases;(A) a 70-year-old man with hearing loss and hypertension or (B) a 60-year-old man with hypertension and financial stress. Following the visit, ASPs provided verbal feedback and completed a behaviorally anchored checklist to rate telemedicine and communication skills, chronic disease management, and use of VA resources. Domain summary scores were calculated as the mean percent of “well done” items and compared using a t-test. Individualized reports and educational resources were shared with participants within a week. Post ASP visit, participants reflected on the feedback they received and their experience. RESULTS: 56 visits (40 residents, 16 attendings) were conducted. There was no significant difference in performance between clinician type and between cases for telemedicine and communication skills. Clinicians performed moderately well in communication skills (information gathering (75% well done), relationship development (68%), education and counseling (73%), and patient satisfaction (86%). They performed less well in telemedicine skills (nonverbal behavior (36%), computer etiquette (36%), optimizing technical aspects of video (26%). Use of clinical resources varied between cases. For case A, only 42% placed referrals for mental health counseling but 88% for audiology. For case B, 60% referred to social work but only 21% to a pharmacist for hypertension management. Participants agreed that the experience was engaging (92%) and a good use of their time (88%). When asked what they would do differently, most clinician responses related to optimizing technology (“be more aware of camera angles,” and “ask about volume on the device”), ensuring patient preparedness (“be clear about time limit,” and “not assuming computer literacy”), and adapting relationship building skills to the video platform (“give time for social conversations,” and “comment on patient's home environment”). CONCLUSIONS: Findings suggest ASPs are an effective and engaging educational methodology to assess clinician practices across multiple domains, including key telemedicine skills. ASP feedback facilitated just-in-time training and commentary from both clinicians and ASPs and identified areas for improvement in future telehealth practice and assessment.
ABSTRACT
BACKGROUND: COVID-19 forced health systems to convert to telemedicine almost overnight. In turn, systems, physicians, and leadership are interested in optimizing telemedicine care quality. Most investigations into telemedicine experiences have been singular snapshots of attitudes or competencies. A better understanding of longitudinal experiences will be necessary to inform ongoing integration of telemedicine into practice. We surveyed practicing physicians a year apart to examine their experiences before and after regular provision of virtual care. METHODS: Surveys were distributed May-Jul '20 (following the first COVID wave) and again in Mar-Jun '21 to internal medicine physicians practicing in free-standing and hospital-based outpatient practices in two distinct health systems: NYU Langone Health (private) and NYC Health + Hospitals (public). Of the 391 physicians who received a survey, 136 participants responded (35% response rate);111 responses contained complete data and were included in analyses. Items covered physician experiences with and attitudes towards tele-visits. For analyses, 4-and 5-point Likert scale items were dichotomized (Disagree v. Agree) or grouped into three item responses (More Difficult, Roughly the Same, Easier). Chi-square analyses were used to explore differences by site and assessment year to describe changes between survey periods. Items on attitudes toward the future of remote-based care were collapsed into a mean 'telemedicine acceptance' score. Regression models were run in order to identify which items are associated with telemedicine acceptance. RESULTS: Physicians reported less difficulty with: taking an appropriate history (49% in 2020, 33% in 2021, p=.015);maximizing patient adherence (33% in 2020, 19% in 2021, p=.028);maintaining patient relationships (31% in 2020, 25% in 2021, p=.009);and sharing information with team members (31% in 2020, 27% in 2021, p=.795) in the follow-up survey. They reported continued challenges with: building new patient relationships (75% in 2020, 77% in 2021, p=.075) and working collaboratively (38% in 2020, 41% in 2021, p=.794). On acceptance measures, physicians reported increased satisfaction with tele-visits over in-person visits (13% in 2020, 27% in 2021, p=0.006) and less worry over doing future tele-visits (45% in 2020, 31% in 2021, p=.027). Fewer physicians found telemedicine to be exhausting than previously reported (51% in 2020, 43% in 2021, p=.247). Mean acceptance scores were 2.41 for 2020 and 2.53 for 2021 (p=.130) (with 4 signifying full acceptance). The ability to establish relationships with new patients (B=0.145, p=.039) and take an appropriate patient history (B=0.154, p=.044) during televisits are associated with increased acceptance. CONCLUSIONS: Results suggest that telemedicine care improves for physicians over time following integration into practice and identify areas for ongoing training/improvement. Long-term follow-up can build a deeper understanding of the ongoing use of telemedicine.
ABSTRACT
BACKGROUND: Since the onset of the COVID-19 pandemic, healthcare systems have faced significant barriers to providing quality primary care, particularly as practices shifted to telemedicine modalities without established technical and educational frameworks for patients, teams, and clinicians. We created an iterative quality improvement project with Unannounced Standardized Patients (USPs) to explore variation in telehealth practices across three public ambulatory care clinics. METHODS: Clinical leadership designed two USP cases reflective of local patient populations and their common clinical needs. USPs portrayed either;(1) a 40-45-year-old Black male with hypertension, or (2) a 40-45-year-old Latina with an asthma exacerbation and hypertension. Both were vaccine hesitant. USPs evaluated visit workflow and clinician's communication skills across core domains (Table 1). After each visit, the USPs completed a behaviorally anchored checklist. Domain summary scores were calculated as mean percent marked “well done.” A t-test was used to compare scores across phases and cases. RESULTS: 60 visits (48 video, 12 audio-only) were conducted in two phases (May-August 2021;September-December 2021). Of the 24 USPs (18 calls, 9 texts) contacted prior to their visit, only 4 spoke directly to a care team member. 74% of USPs recommended the clinic. There were no significant differences in domain scores between phases or cases (Table 1). Most clinicians (82% in both phases) introduced the topic of the COVID-19 vaccine appropriately. Regarding screening, most providers asked about smoking (79%) and alcohol use (72%), but few screened for vaping (22%) or depression (4%). 70% of clinicians or care teams replied to a MyChart portal message that was sent by the USP to the care team after the visit. CONCLUSIONS: Findings highlight opportunities for system-based change to optimize telehealth care (particularly the integration of team members in previsit planning, standardized screenings, and patient follow-up). Data across phases indicate sustained need for quality improvement efforts;reviewing comparative data with clinic leadership will inform further evaluation of health systems and educational methods.
ABSTRACT
BACKGROUND: Student health centers provide necessary care to university students, yet there is limited literature on efforts to standardize or improve quality of this care. When our urban student health center rapidly shifted to telemedicine-based care at the onset of COVID-19, we designed and deployed two unannounced standardized patient (USP) cases as part of a quality improvement project to assess, broadly, the impact of a new telemedicine workflow and common clinical activities. METHODS: In April and May of 2021, 12 primary care and 4 women's health clinicians conducted virtual visits with two distinct USP cases (N=32 visits, 2 per clinician). Cases included (1) a 21-year-old female requesting a birth control prescription with a positive PHQ-9 and (2) a 21-year-old gaymale who has questions regarding safe sex and vapes regularly. Clinicians were evaluated using two assessments: (1) a behaviorally anchored checklist completed by the USP covering the microsystem and the clinician's skills and (2) a systematic chart review of the electronic health record. RESULTS: Overall, 88% of USPs reported that they would recommend the clinic. Regarding the clinic workflow, 12 of 32 USPs reported receiving a previsit call (clinical protocol), 10 answered the calls and of those, 20% verified two patient identifiers. Between cases, there were significant differences in three domains (mean % well done). Information gathering was significantly higher for the depression case (84% vs. 48%;p=0.004), patient activation was higher for the sexual health case (53% vs 23%;p=0.027), and telemedicine skills were higher in the depression case (77% vs. 57%;p=0.015). Clinician performance was relatively low in case-specific education (depression: 28%, sexual health: 20%) and moderate in management/treatment plan (mean across cases: 66%), relationship development (64%), and education and counseling (65%). In a number of “gold standards of care” items identified by clinical leadership, less than half of clinicians met the target. In the depression case, 56% documented both PHQ-2 and PHQ-9 screenings, 0% screened for alcohol use and quantity, and 66% screened for relationship violence. In the sexual health case, 56% screened for vaping quantity and frequency. 63% counseled on PrEP. No providers completed any part of a physical exam besides commenting on appearance. CONCLUSIONS: Through USP visits and chart reviews, we identified variation in quality of communication skills and use of common screening protocols at this student health center during virtual visits. Health system standards previously integrated into in-person visits may have been lost in the transition to telemedicine care. Our results suggest opportunities exist for targeted improvement efforts that ensure quality virtual care for all students.
ABSTRACT
SETTING AND PARTICIPANTS: Clinician trainees across our health system, including: 1) 107 internal medicine faculty and residents who participated in workplace-based learning at public, private, and federal (Veterans Affairs) ambulatory practices, 2) 16 clinicians at our student health center, and 3) upwards of 250 medical students, residents, and newly-hired general internal medicine (GIM) faculty members from medicine, neurology, and pediatrics departments in our simulation center. DESCRIPTION: While core communication skills have always been at the forefront of medical trainee assessment, information on transference of those skills and integration of the in-person clinical workflow to the virtual care environment was limited prior to COVID-19. NYU Grossman School of Medicine (NYUSOM) implemented a telehealth improvement program across medical students, residents and faculty. In order to assess and improve our systems' ongoing telehealth practices, we employed three distinct educational methodologies across our health systems since March 2020: objective structured clinical exams (OSCEs) and announced (ASPs) and unannounced standardized patient (USPs). Cases were designed to target common, site-specific issues (i.e., hearing loss, COVID-19 vaccine hesitancy, social determinants of health, and sexual and mental health concerns). In line with previous work, all SPs were trained to use a standard behaviorally-anchored checklist to assess communication and telemedicine-specific skills over video visit ( Zoom or WebEx). USPs, professional actors who conduct visits unbeknownst to the clinician, were also trained to collect data on clinic functioning. EVALUATION: Summary reports on performance were provided to both clinical and education leadership and learners to identify future training needs. Data on telemedicine skills across all projects demonstrates room for improvement (mean % marked 'well done' across learners: 46% in the OSCE, 68% at the SHC, and 48% in the public clinics, respectively). Common telemedicine challenges included prompting the SP to adjust their video frame or remove distracting background noise. Most health systems conducted fewer screenings virtually than they did inperson (e.g., at the SHC only 41% and 6% of SPs were screened for alcohol and vaping, respectively;at the public clinics, 25% and 20% were screened for depression and vaping, respectively). Participant feedback reports highlight performance across core domains and provide resources for improvement. DISCUSSION / REFLECTION / LESSONS LEARNED: Our ongoing telemedicine training program demonstrates a highly scalable educational assessment methodology that can be leveraged to optimize common care practices. Data confirm that SPs, ASPs, and USPs can be used across the health care system in simulated and real-world scenarios to identify areas for intervention.
ABSTRACT
BACKGROUND: The COVID-19 pandemic has led to an unprecedented and rapid increase in telemedicine visits. It is unclear how this has affected resident training. This study explores Internal Medicine residents' perceptions of the impact of the transition to telemedicine on their learning experience. METHODS: Three focus groups were held over Zoom with post-graduate year 1 - 3 residents at the New York University Grossman School of Medicine who have their Primary Care continuity experiences at the Manhattan Veterans Affairs Medical Center. Issues explored include relationships with patients, decision-making around clinical issues, use of preceptors, and general impressions of tele-visits. Recordings of the focus groups were transcribed and thematically analyzed. RESULTS: Common themes across the focus groups were categorized into five domains: 1) building patient relationships, 2) using the remote setting, 3) judging appearance and physical examination, 4) confidence with decision making, and 5) interactions with supervisors. Participants felt that they were able to maintain relationships with pre-existing patients, particularly those with chronic conditions for whom tele-visits were an appropriate and even preferable alternative to routine in-person encounters. Participants reported difficulty connecting and building rapport with new patients but suggested the remote setting was helpful for providing additional context (e.g. input from a family member), allowing them to better understand patients' concerns or circumstances. However, some participants expressed hesitancy about asking personal questions because patients were not in private settings. The inability to conduct comprehensive physical exams made it difficult to evaluate physical concerns, and participants were frequently unable to rely on skills that were emphasized in their training, such as assessing overall appearance and frailty. As a result, they had a higher level of uncertainty about diagnosis and management and a lower threshold for making referrals and ordering diagnostic testing. Residents did not ask preceptors for help during a tele-visit - in contrast to in-person clinical visits - which precluded real-time support when participants encountered uncertainty. Due to the awkwardness of asynchronous communication, participants had a higher threshold for asking their preceptors questions and making changes in management plans after televisits were concluded. CONCLUSIONS: In light of the staying power of telemedicine and the overall trend towards virtual care, our findings substantiate the need to identify and address new clinical training competencies. In particular, trainees may need guidance for negotiating clinical decision-making and management when caring for patients remotely;structured preceptor protocols can help to support trainee education and quality patient care.
ABSTRACT
Introduction: The objective structured clinical examination (OSCE) has been shown to not only assess but also improve the performance of trainees. Our group has previously demonstrated the benefits of OSCEs to assess gastroenterology (GI) fellows. We have successfully assessed performance across numerous milestones. Typically, OSCEs are held in person, however the COVID-19 pandemic has precipitated the need for virtual learning. We accordingly transitioned to a virtual zoom OSCE (Z-OSCE) and evaluated trainees' perception of this program. Methods: Fourteen first- and second-year GI fellows from five programs across multiple states participated in a four-station virtual OSCE on Zoom. Afterwards, participants answered a survey to share their perspectives and provide feedback. Learners were asked to rate the usefulness of the virtual OSCE and compare it to other in-person and virtual educational modalities. These questions were rated on a 10-point Likert scale (Figure 1). Additionally, free-text responses regarding any aspect of the OSCE were evaluated for comments on the virtual format. Results: In comparing the usefulness of the virtual OSCE to other in-person modalities, trainees rated it a mean of 7.15 (range 5-10), and 31% of respondents rated it a 9 or 10. Trainees rated the virtual OSCE compared to other virtual learning modalities a mean of 8.15 (range 5-10), and 43% rated it 9 or 10. When asked whether they would recommend this OSCE as a training tool, the trainees gave a mean recommendation of 7.77 (range 5-10), and 38% gave a 9 or 10. General feedback regarding the nature of the OSCE noted the virtual format worked well, orientation to the format was important and could be improved by providing it in an email beforehand. Conclusion: Virtual learning has been necessary during the COVID-19 pandemic, and it is crucial to evaluate the value of the novel Z-OSCE. Participants found the virtual OSCE may be more useful than in-person learning modalities and it compared favorably to other virtual learning modalities. One benefit of this modality was the easier inclusion of fellows from geographically disparate areas negating the need to travel for this program, a benefit given lack of universal access to simulation using standardized patients. To improve future exams, orientation prior to the day of the OSCE may improve trainees' experiences..